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ABSTRACT 

ANALYSIS  OF  THE  TRANSFER  OF  TRAINING,  SUBSTITUTION,  AND 
FIDELITY  OF  SIMULATION  OF  TRAINING  EQUIPMENT 

This  report  summarizes,  evaluates  and  synthesizes  the  data  on  the 
training  value  of  training  devices.  The  report  discusses  the  issues 
of  substitution  of  some  operational  training  time  by  training  devices 
and  the  relationship  betvreen  training  effectiveness  and  cost  (fidelity 


of  simulation) . 


FOREWORD 


This  report  was  prepared  as  one  element  of  a  staff  study  en  coat 
and  training  effectiveness  undertaken  by  a  Training  Analysis  and 
Evaluation  Group  (TAEG)  Ceaa.  It  was  prepared  by  Dr.  G&ns  S.  Kicheli. 

The  min  report  prepared  by  the  TAEG  team,  of  which  this  repofffc  to 
a  supplement,  .to  entitled,  "Staff  Study  on  Cost  and  Training  &£fe$gi*r®Ses3 
of  Proposed  Training  System"  (NAVTEAEQUSPCEN  TAEG  deport  1), 

The  report  la  separately  published  because  it  ada’reaaes  en  issue 
which  hae  relevance  not  only  for  the  Staff  Study  on  Coot  end  Training 
Effectiveness  of  Proposed  Training  Systems  (NAVTRAEQUIPCEN  TAEG  Report  1) , 
but  also  for  the  broader  questions  of  training  equipment  (de-vice)  fidelity 
of  simulation  and  the  substitution  of  training  device-based  training  for 
training  using  operational  equipments  and  environments. 


ANALYSIS  OF  THE  TRANSFER  OF  TRAINING,  SUBSTITUTION,  AND 
FIDELITY  OF  SIMULATION  OF  TRAINING  EQUIPMENT 

"If  we  could  first  know  where  we  are,  and  whither 
we  are  drifting,  we  could  better  judge  what  to 
do  and  how  to  do  it."  -  Abraham  Lincoln 

The  purpose  of  this  reiport  is  to  analyze  the  current  situation  on 
the  cost  and  training  effectiveness  of  training  devices. 

The  cost  and  training  effectiveness  of  training  devices  are  ideally 
determined  by  the  collection  of  empirical  data  by  controlled  experiments. 
At  present,  however,  there  is  a  paucity  of  such  data.  What  does  exist 
will  be  summarized  in  terms  of  the  commonality  of  findings.  Specific 
current  training  situations  will  be  analyzed  to  determine  tasks  which 
can  be  learned  in  the  training  system  and  in  the  operational  situation. 

From  the  results  of  the  analyses  of  specific  transfer/sufestltution 
studies,  an  attempt  will  be  made  to  generalise  to  various  types  of 
training  situations  in  order  to  arrive  at  an  evaluafciqn  or  the  cost  and 
training  effectiveness  of  training  devices. 

"Cost  effectiveness"  will  be  used  in  this  paper  to  mean  the  yae  of 
the  least  costly  of  several  alternative  training  systems,  all  of  which 
could  equally  produce  men  trained  to  a  specified  level  of  proficiency. 
Lower  cost  of  training  equipment  allows  (even  demands)  its  use  in  place 
of  operational  equipment. 

The  "training  effectiveness"  of  a  training  device  is  usually 
expressed  as  a  measure  of  transfer  of  training.  Transfer  of  training 
refers  to  the  degree  to  which  practice  in  a  trainer  carries  over  to 
(or  affects)  performance  in  ar«  operational  situation,  as  compared  to  the 
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performance  of  trainees  who  received  no  practice  in  the  trainer..  In 
other  words,  training  effectiveness  is  the  difference  between  a  performance 
measurement  on  an  operational  task  after  practice  on  the  training  device 
and  performance  on  the  operational  task  without  practice  on  the  training 
device.  (Trainees  who  receive  practice  in  a  training  device  are  usually 
referred  to  as  the  experimental  group;  trainees  who  receive  no  practice 
in  the  trainer  are  referred  to  as  the  control  group.) 

Thus,  transfer  of  training  is  the  term  used  to  describe  how  what  is 
done  (learned)  in  one  situation  affects  what  is  done  (performance)  in 
another.  Transfer  of  training  is  positive  when  a  training  situation 
aids,  subsequent  performance.  It  is  negative  when  it  hinders  that 
performance*  and  it  is  zero  when  training  has  no  effect  on  later 
performance. 

Moat  measures  of  training  effectiveness  are  measures  of  transfer 
of  training.  Many  different  formulas  exist  for  expressing  the  amount 
of  transfer  (References  1,  2,  and  3). 

Percent  transfer  based  on  improvement  in  performance  on  the 
operational  task  or  on  savings  in  time  to  reach  a  specified  performance 
level  on  the  operational  task  may  be  calculated  from  the  following  formula: 

%  transfer  «*  Zc  -  Ze  x  100 
Zc 

Where: 

Zc  *  performance  or  time  required  on  the  operational  (or  transfer) 
task  by  the  control  group. 

Ze  *  the  corresponding  value  for  the  experimental  group. 
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The  Northrop  Air  Force  Future  Undergraduate  Pilot  Training  System 
Study  (Ref.  4)  equated  the  percent  transfer  formula  (based  on  savings 
in  time)  with  "replacement  percent"  to  denote  that  it  is  an  index  of 
the  percent  of  time  on  the  operational  task  which  can  be  saved  or 
"replaced"  by  time  in  the  training  device. 

Roscoe  of  the  University  of  Illinois  Aviation  Research  Laboratory 
believed  that  the  quantitative  assessment  of  the  transfer  of  training 
from  training  devices  to  operational  tasks  is  not  adequately  described 
by  the  percent  transfer  measure.  The  fact  that  trainers  result  in  & 
saving  of  time  to  reach  a  specified  level  of  performance  in  the  opera¬ 
tional  task  is  meaningful  only  if  the  time  in  the  trainer  is  known. 

This  resulted  in  the  development  of  J:e  transfer  effectiveness  ratio 
(TER)  (Ref.  5).  The  TER  is  a  measure  for  assessing  the  effectiveness 
of  a  training  device  by  expressing  the  saving  in  time  on  the  operational 
task  as  a  function  of  the  amount  of  time  in  the  trainer.  It  is  defined 
as  time  saved  in  the  transfer  (or  operational)  task,  divided  by  the 
time  required  in  the  training  device.  The  TER  may  be  calculated  from 
the  following  formula: 

TER  «  Yc  -  Ye 
Xe 

Whera: 

Yc  «=  tin.e  required  by  the  control  group  to  reach  some  Criterion  of 
proficiency  in  the  operational  (or  transfer)  task 

Ye  =  the  corresponding  value  for  the  experimental  group. 

Xe  «  the  training  device  hours  received  by  the  experimental  group. 

Examples  of  calculations  of  percent  transfer  and  TER  follow. 
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From  data  in  Reference  6: 

%  Transfer  «  60  hrs.  -  6.5  hrs.  x  100  ®  89% 

60  hrs. 

TER  =  60  hrs.  -  6.5  hrs.  =1.25 
42.8  hrs. 

From  data  in  Reference  7: 

%  Transfer  =  211  hrs.  ~  18  hrs.  =  91%  (Flight  check  criterion) 

211  hrs. 

TER  »  211  nrs.  -  18  hrs.  =  .91  (Flight  check  criterion) 

213  hrs. 

%  Transfer  »  262  hrs.  ~  113  hrs.  »  57%  (Criterion  of  completing 
262  hrs.  B~Stage) 

TER  «  262  hrs.  ~  113  hrs.  =  .57  (Criterion  of  completing  B-Stage) 
261  hrs. 

These  data  show  that  the  same  training  device  may  exhibit  different 
transfer  effectiveness  ratios  depending  upon  the  criterion  of  performance 
used.  Also,  far  different  stages  of  a  curriculum,  a  training  device  may 
have  different  TER’s.  And,  of  course 9  the  effectiveness  of  a  training 
device  depends  greatly  on  how  it  is  used.  However,  although  the  value 
of  measures  of  effectiveness  may  change,  they  are  useful  for  studying 
learning  and  transfer. 

A  number  of  transfer  of  training  experiments  have  beju*  performed 
which  demonstrate  that  trainers  can  be  used  effectively  to  reduce 
operational  (e.g.,  flight)  training  time  by  significant  proportions. 

(See  Appendix  A  i'er  summaries  c£  training  system  effectiveness  studies.) 

These  transfer  of  training  experiments  lead  to  the  following 
conclusions : 

1.  Simulators  have  cost  and  training  value  for  pilot  training,  since 
they  permit  the  learning  of  flight  asks  in  them.  In  fact,  substantial 
amounts  of  simulator  time  can  be  used  in  place  of  flight  time. 


2.  Most  experimental  work  has  been  done  on  simple  aircraft  and 
trainers,  but  similar  results  have  been  obtained  when  complex  aircraft 
and  trainers  have  been  used. 

3.  Different  kinds  of  flight  tasks  have  different  transfer  effects. 
Simulators  are  best  for  procedural  and  instrument  flying  tasks.  Complex 
maneuvers  have  not  been  learned  as  well  with  the  past  state-of-the-art 
in  simulation. 

4.  How  a  device  is  used  may  influence  learning  and  transfer  to  a 
greater  degree  than  trainer  design. 

Most  of  the  studies  were  conducted  during  the  1940's  and  early 
1950’s.  Sim?  f-:r  research  has  recently  resumed.  The  University  of 
Illinois  A  •  Research  Laboratory  found  in  1971  (Ref.  5)  that  eleven 

hours  .  f  training  in  the  old  ’’Blue  Box"  (AN-T-18)  resulted  in  a  savings 
of  nine  hours  of  flight  time  (out  of  46  hours)  on  the  Piper  Cherokee 
(transfer  effectiveness  ratio  of  0.8).  Eleven  hours  of  training  in  the 
GAT-1  resulted  in  a  savings  of  eleven  hours  of  flight  time  (transfer 
effectiveness  ratio  of  1.0). 

HuraKRO,  in  1971  (Ref.  6),  conducted  an  evaluation  of  the  Synthetic 
Flight  Training  System  (SFTS),  Device  2B24,  using  Army  helicopter 
trainees  who  had  just  completed  110  hours  of  Primary  training  on  the 
TH-55  and  were  ready  to  start  instrument  training  (TH-13T).  Compared 
to  conventionally  trained  students  who  spent  60  hours  in  the  aircraft, 
the  SFTS  trainees  spent  42.8  hours  in  the  trainer  and  6.5  hours  in  the 
aircraft.  A  TER  of  1.25  was  calculated  from  the  data  available  in  the 
report.  Also,  calendar  time  was  eight  weeks,  versus  12  weeks  for  the 
conventional  program. 
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The  NAVTRAEQUIPCEN  conducted  a  transfer  of  training  experiment 
in  1971  of  Device  2F9G,  the  TA-4J  OFT  (Ref.  7).  The  effectiveness  of 
the  device  for  training  on  the  basic  instrument  portion  of  the  advanced 
jet  syllabus  was  evaluated  by  comparing  groups  given  different  training 
regimes.  Three  experimental  groups  were  compared  te  each  other  and  to 
a  control  group  which  had  received  the  standard  syllabus  training.  Of 
the  three  experimental  groups,  one  received  training  only  in  flight, 
another  group  only  in  the  trainer,  and  th*  third  received  only  academic 
training  on  related  principles  of  the  basic  instrument  portion  of  the 
syllabus.  All  groups  were  given  a  flight:  check  in  the  TA-4J  aircraft 
after  training.  Following  the  flight  check,  students  were  recycled  for 
as  many  flights  (in  the  aircraft  or  in  the  trainer)  as  was  estimated 
by  the  check  pilot  to  be  necessary  to  make  them  equivalent  to  those 
receiving  the  standard  syllabus. 

The  results  of  the  experiment  are: 

1.  The  flight  check  scores  of  the  control,  flight,  trainer,  and 
academic  groups  were  3.12,  3.03,  2.99,  and  2.77,  respectively.  The 
control  group  was  best;  however,  there  was  no  statistically  significant 


difference  between  the  flight-trained  and  the  trainer-trained  groups. 

2.  Even  after  the  students  were  recycled  for  as  many  sessions  in 
the  aircraft  and  in  the  trainer  as  the  check  pilots  thought  they 
required,  the  trainer  group  saved  4.7  flight  hours  (or  three  flights) 
compared  to  the  control  group.  However,  the  trainer  group  required  an 
additional  1.6  hours  (or  one  session)  on  the  trainer.  This  is  a  55% 
aavings  in  flight  hours,  which  translates  into  a  considerable  savings 
in  cost  per  trainer  group  student. 
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3.  Calculations  for  transfer  effectiveness  ratios  resulted  in  the 
vainly  0.91  and  0.5/,  depending  upon  the  definition  of  criterion 
performance  used.  The  TER's  are  interpreted  to  mean  that  for  the 
portion  of  the  syllabus  experimented  with,  trainer  sessions  are  almost 
equivalent  to  aircraft  flights  in  trair  .‘ng  effectiveness,  or  have  an 
equivalent  value  of  0.57  to  1,  depen:,  ng  on  whether  is  used  the  hours 
required  to  pass  the  flight  check  or  the  hours  to  complete  the  basic 
instrument  stage  (which  includes  recycled  flights  in  the  aircraft  and 
trainer) „ 

In  an  attempt  to  provide  a  common  basis  for  comparing  the  results 
of  different  studies,  percent  transfer  based  on  a  savings  in  time 
(replacement  percent  according  to  Northrop)  and  TER’s  were  computed  by 
Northrop  (Ref.  4).  The  t&aka  trained  were  contact  flight  procedures 
and  maneuvers,  landing,  and  takeoff.  The  percent  transfer  and  Training 
Effectiveness  Ratio  for  various  studies  are  given  in  Table  1. 

The  preceding  data  convincingly  demonstrate  that  flight  simulator 
training  transfers  to  aircraft  and  can  be  substituted  for  some  flight 
time.  This  is  true  for  civilian  pilots  of  light  aircraft,  military 
undergraduate  pilot  training,  and  airline  pilot  transition  training. 
Further  evidence  of  the  transfer  and  substitutability  of  flight  training, 
of  course,  is  NASA's  Apollo  Program  in  which  100%  of  the  training  for 
space  flight  and  lunar  landings  was  conducted  in  simulators. 

Apparently  it  is  not  true,  as  is  believed  by  some,  that  the  airlines 
can  conduct  most  (and  eventually  100%)  of  its  transition  training  in 
simulators  only  because  their  pilots  are  very  experienced.  The  studies 
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TABLE  !•  PERCENT  TRANSFER  AND  TER'S  CALCUIATSD  FOR  SEVERAL  STUDIES 

(After  Reference  4> 
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reported  above  provide  sufficient  evidence  ttxat  flight  training  devices 
are  also  effective  for  neophyte  pilot  training. 

Some  discussion  is  warranted  here  about  the  experience  of  the 
airlines.  They  have  followed  a  traditional  pattern  for  many  years. 

Each  training  program  was  modeled  after  earlier  ones  with  very  little 
change.  Pilots  practiced  maneuvers  in  the  airplane  to  develop  skill 
needed  for  passing  a  rating  check.  Accidents  occurring  in  training 
flights  while  practicing  high  risk  maneuvers  and  particularly  the 
prohibitive  cost  per  flying  hour  has  forced  the  airlines  to  use  simu¬ 
lators  for  transition  training. 

The  airlines  have  performed  detailed  task  analyses  as  a  basis  for 
defining  Specific  Behavioral  Objectives  (SBO's)  to  restructure  their 
training  programs  to  make  maximum  use  of  simulators.  For  example, 
American  Airlines  has  gone  from  20.6  hours  on  its  B727  aircraft  in  1966, 
to  7.6  hours  in  1969  (63%  reduction  of  flight  training  hours).  Check- 
rides  by  F/tA  Flight  Standards  Inspectors  have  demonstrated  that  pilots 
trained  using  a  ratio  of  28.2  flight  hours  to  7.6  simulator  hours  can 
qualify.  American  Airlines’  goal  is  to  achieve  100%  training  in 
simulators  (due  to  cost).  Its  DC-10  transition  training  program  is 
currently  two  hours  of  flight  time. 

In  the  military s  also,  training  devices  have  been  used  as  supplements 
to  flight  training.  For  the  following  reasons,  however,  there  is  little 
choice  but  to  substitute  for  flight  time  by  training  devices  (Ref.  16): 

Cost:  The  complexities  of  current  and  future  aircraft  and  weapon 
systems  are  driving  the  cost  per  flying  hour  to  such  a  level  that  all 
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but  "payload"  or  "mission"  flight  is  prohibitively  expensive.  On  this 
basis,  alternatives  to  present  concepts  of  flight  training  demand 
investigation,  development,  and  implementation  to  provide  adequately 
skilled  aircrews. 

Air  Space;  The  speeds  attainable  by  current  and  future  aircraft 
require  greater  operating  air  space  per  unit  than  that  known  in  the  past. 
This  fa:t,  considered  with  the  increasing  demands  for  air  service,  places 
a  premium  upon  an  already  overloaded  air  space.  When  saturation  occurs, 
catastrophe  may  be  the  result.  The  implications  for  flying  safety,  as 
well  as  jfficient  operations,  are  readily  apparent.  Again,  alternatives 
to  presert  practices  and  procedures  must  be  developed  to  better  use  and 
conserve  this  fixed  resource  of  air  space. 

Flying  Safety;  Each  flight  in  an  aircraft  is  an  exposure  to  danger, 
however  smell.  Training  flights,  in  addition,  expose  the  trainee  during 
the  period  he  is  least  capable  of  coping  with  dangerous  or  emergency 
occurrences.  The  vclue  of  human  life  is,  of  course,  incalculable  as  it 
has  always  bev.n.  The  value  per  unit  of  current  and  future  aircraft  is 
such  that  the  financial  penalty  for  losing  an  aircraft,  when  alternatives 
can  be  made  available,  is  too  great  to  be  justified. 

Data  such  as  presented  above  have  apparently  convinced  Navy  and 
Air  Force  planners  that  flight  substitution  is  feasible.  Consideration 
is  being  given  to  the  substitution  recommendations  of  the  UPT  studies. 

The  Navy  study  (North  American  and  Link,  Ref.  17)  of  undergraduate 
pilot  training  states  45%  substitution  will  be  possible  overall  when 
a  u'.de  angle  visual  for  operational  flight  trainers  (OFT's)  becomes 
i  .*« liable.  The  Air  Force  UPT  studies  state  that  50%  (Northrop,  Ref.  4) 
and  47%  (Lockheed,  Ref.  13)  will  be  possible. 
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The  Navy  UPT  (North  American/Link)  recommendations  for  substitution 
of  flight  time  by  ground  training  devices  were  based  on  an  analysis  of 
the  tasks  involved  in  flight  training.  These  tasks  were  analysed  to 
identify  the  kinds  of  learning  processes  involved,  and  to  identify  the 
kinds  of  demand  they  tend  to  make  on  the  training  setting.  It  was  found, 
for  example,  that  most  instrument  flight  trainers  are  highly  procedural, 
and  require  primarily  that  the  training  environment  contain  accurate 
representations  of  cockpit  displays  and  controls.  Other  maneuvers, 
aerobatics  for  example,  while  containing  significant  procedural  elements, 
also  require  pilot  surveillance  cf  a  variety  of  out-of-the-window  visual 
cues  to  position,  attitude,  heading,  and  airspeed.  Because  of  the 
relative  ease  of  simulating  events  represented  in  cockpit  displays, 
motions  and  sounds,  primary  attention  in  allocating  training  tasks  to 
simulation  has  been  given  to  requirements  for  the  representation  of  out- 
of-the-window  visual  information.  Each  of  the  non- instrument  flight 
maneuvers  trained  in  the  undergraduate  program  requires  out-of-the- 
window  visual  information.  Some  maneuvers  require  simple  cues  which 
are  readily  provided  at  reasonable  expense.  Others  require  more 
complex  cues,  at  greater  cost.  A  few  require  visual  cues  which  have 
such  extensive  equipment  implications  that  their  incorporation  in  ground 
training  devices  would  be  uneconomical  within  the  undergraduate  program.. 
In  allocating  flight  tasks  to  ground  trainers,  consideration  was  given 
to  the  relative  expense  of  ground  and  flight  training,  to  assure  the 
allocation  of  tasks  to  ground  training  which  would,  in  fact,  represent 
significant  cost  savings.  Two  hypothetical  simulators.  Simulator  "A" 
and  Simulator  "B",  were  conceived  in  making  tentative  allocations. 


Training  devices  using  the  Simulator  "A"  concept  would  have  a  three- 
degree-of-freedom  motion  system,  and  a  visual  system  display  CRT  w^uld 
have  a  48°  horizontal  and  a  28°  vertical  field  of  view.  Cockpit 
controls  and  instruments  would  have  the  same  extent  of  fidelity  in 
current  military  ant  commercial  flight  simulators.  Other  features,  such 
as  task  and  maneuver  demonstrations,  performance  measurement,  feedback, 
and  permanent  recordings  of  performance,  would  be  included.  Training 
devices  employing  the  Simulator  "B"  concept  would  have  a  180°  horizontal 
and  87°  vertical  field  of  view  and  will  include  a  generalized  earth 
(or  sea),  and  sky  with  horizon.  It  will  have  a  six-degree-of-freedom 
motion  system.  Other  characteristics  will  be  similar  to  those  described 
in  the  "A"  concept. 

A  set  of  ground  rules  was  developed  by  North  American  and  Link  to 
guide  the  estimation  of  the  relative  effectiveness  of  various  allocations 
of  flight  tasks  to  ground  training  devices.  The  extent  of  substitution 
of  training  device  time  for  flight  time  resulted  from  the  application 
of  these  ground  rules.  In  defining  ground  rules  for  the  reduction  of 
flight  hours  through  the  use  of  ground  simulation,  the  recent  training 
literature  was  reviewed  for  empirical  evidence  of  successful  substitutions 
of  device  for  aircraft  time.  Also,  analyses  were  made  of  the  maneuvers 
to  be  trained  in  the  recommended  program  and  of.'  the  simulator  capabilities 
available  for  supporting  this  training,  to  define  the  pilot  task 
elements  to  which  ground  training  is  applicable.  Device  design  concepts 
were  developed  to  incorporate  these  task  element-related  capabilities 
based  on  the  identification  of  task  elements  within  the  capabilities 
of  device  concepts.  An  analysis  of  flight  tasks  likely  to  be  employed 
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in  training  naval  pilots  in  the  future  program  identified  the  kinds  of 
learning  functions  involved  in  each  task.  This  analysis  was  used  to 
anticipate  the  types  of  training  setting  most  appropriate  to  these 
functions,  and  estimates  were  made  of  the  extent  to  which  each  task 
depends  on  each  function.  Guidelines  were  developed  for  assigning 
percentages  of  flight  tasks  to  training  devices: 

a.  Substitution  of  simulator  for  aircraft  flight  time  is  determined 
primarily  by  the  proportion  of  maneuver,  or  task  training  time,  which 
would  be  devoted  in  flight  to  learning  procedural  or  fixed-sequential 
task  elements.  Each  task  involved  in  aircraft  and  system  operation 
contains  a  significant  procedural  component.  That  is,  each  task  requires 
the  selection,  initiation  and  execution  of  some  fixed  sequence  of 
control  outputs  whose  magnitude  and  timing  are  keyed  to  sets  of  relatively 
well-defined  events.  Ground  simulation  is  particularly  effective  in 
training  these  procedural  task  elements  because,  by  definition,  they 
involve  cues  to  control  actions  which  can  be  readily  identified  and,  in 
most  cases,  adequately  represented  in  simulation. 

b.  Approximately  25%  of  the  time  normally  spent  in  solo  flight  can 
be  re-allocated  to  ground  simulation. 

c.  Approximately  75%  of  instrument  and  radio  instrument  flight 
training,  which  does  not  involve  an  out-of-the-window  reference,  can  be 
provided  in  ground  simulation  devices.  The  motion  cue  requirements  in 
almost  all  of  the  procedural  and  skill  elements  of  instrument  flight 

are  well  within  the  state-of-the-art,  and  only  minimal  visual  requirements 
exist  in  ihis  training  stage.  Near -perfect  fidelity  of  instrument 
display  and  control  representation  is  readily  available,  making  ground 
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simulation  almost  totally  equivalent  to  actual  instrument  flight. 

The  primary  discrepancy  is  in  the  stress  involved  in  actual  aircraft 
flight,  and  the  knowledge  that  inaccurate  performance  can  have  serious 
consequences.  For  this  reason,  it  is  essential  that  skills  learned  in 
the  unstressed  ground  trainer  environment  Be  demonstrated  in  actual 
flight. 

d.  Approximately  50%  of  dual  flight  time,  not  involving  motion 
and  visual  capabilities  outside  the  simulation  state-of-the-art,  can 
be  provided  in  ground  simulation.  The  superior  capability  of  the  simu¬ 
lator  for  permitting  instructor  monitoring  of  student  performance,  the 
capability  for  practicing  maneuvers  and  procedures  which  could  not  be 
practiced  in  the  air,  and  the  simulator  capability  for  concentrating 
only  on  training-relevant  task  elements  contribute  significantly  to 
its  ability  to  substitute  for  dual  instruction  time. 

An  example  of  recommendations  made  for,  substitution  of  simulator 
time  that  may  be  made  for  flight  time  in  the  Advanced  Jet  syllabus  is 
shown  in  Table  2,  which  is  taken  from  a  draft  of  Ref.  17,  dated  April  1971. 

If  the  Navy  and  Air  Force  future  DPT  studies'  analyses  and  conclusions 

that  as  much  as  50%  (and  75%  in  some  specific  areas)  of  flight  time  can 

be  substituted  for  by  simulators  soem  high,  consider  the  viewpoints 

of  participants  in  a  symposium  on  pilot  training  and  the  pilot  career 

% 

conducted  by  the  lUmd  Corporation  (Ref.  19). 

In  discussing  the  question,  "How  far  do  all  present  want  to  go  with 
ground-based  simulation  in  substitution  for  aircraft?",  a  representative 
of  the  University  of  Illinois  Institute  of  Aviation  proposed  aiming  for 
1Q0%.  Re  projected  an  answer  for  the  commercial  airlines,  which  would  be 
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STAGE 

SYLLABUS 

HOURS 

SIM  A 
SUB  HRS 

SIM  A 
SUB  % 

j  SIH  B 

SU3  HRS 

j  SIM  B  ! 

SUB  % 

Familiarization 

11.2 

2.8 

25 

5.6 

50 

Bade  Xnsts, 

8,4 

5.6 

67 

5.6 

67 

Inst,  Nav , 

36.1 

29.8 

82 

23.6 

82 

Fonaation 

12.6 

0.0 

0 

2.8 

22 

Night  Flying 

8.5 

1.5 

18 

5,6 

67 

Oper  Nav 

9.3 

0,0 

0 

4.0 

43 

Appl.  Nav 

4.5 

3.0 

6t- 

3.0 

66 

Air/Grnd  Wpns 

12.1 

0.0 

0 

5o5 

45 

Tactics 

9.9 

0.0 

0 

4.4 

44 

Air/Air  Wpns 

4.0 

0.0 

0 

1.0 

25 

Car  Qua! 

12.4 

1.6 

4.8 

.J2  ! 

129.0 

44.1 

71.9 

56%  1 
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to  take  pilots  straight  from  the  simulator  to  flying  the  aircraft.  A 
commercial  airlines  representative  confirmed  that  for  the  reason  of  cost, 
the  airlines*  objective  is  to  perform  100%  training  in  simulators. 

A  representative  of  the  USAF  Human  Resources  Laboratory  stated 
that  a  100%  goal  is  a  meaningful  and  viable  goal  provided  that  constraints 
operating  to  prevent  its  achievement  are  recognized.  Such  constraints 
are:  cost,  probability  of  not  achieving  100%  fidelity,  stress,  moti¬ 
vation,  and  joy  of  flying.  In  his  opinion,  the  100%  goal  is  not  an 
unreasonable  aspiration  under  such  conditions. 

Most  of  the  work  was  done  with  pilots,  probably  because  of  the 
desire  for  information  in  such  a  high  risk  activity.  Another  reason 
may  be  that  the  feasibility  of  substitution  is  readily  apparent.  Even 
with  the  paucity  of  transfer  data  in  other  areas,  however,  there  is 
convincing  justification  for  use  of  training  device  substitution.  Further* 
substitution  could  be  started  prior  to  the  collection  of  empirical 
evidence  in  many  cases  because  of  the  relatively  small  risk  involved. 

Studies  of  non-flying  activities  in  which  transfer  data  was  obtained 
follow.  Device  3A105,  Tracked  Vehicle  Driving  Trainer  (M48A3  tank), 
was  evaluated  at  the  Tracked  Vehicle  School,  Camp  Pendleton  (Ref.  20). 

It  was  found  that  driver  training  using  the  trainer  was  as  effective  as 
training  on  the  actual  tank.  Training  was  accomplished  in  approximately 
an  equal  amount  of  time  using  the  trainer  versus  using  the  actual  tank. 
Since  the  time  differences  between  those  groups  using  the  trainer  and 
those  using  the  tank  alone  are  almost  equal,  it  was  concluded  that  there 
was  a  one-to-one  replacement  ratio  between  the  trainer  and  the  tank 
itself  as  far  as  driver  training  is  concerned.  Further,  since  there  is 
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a  significant  cost  differential  between  the  twi  modes  of  training,  the 
training  device  represents  a  more  cost~effect:'.ve  method  of  training 
tank  drivers. 

An  evaluation  of  the  Carrier  Air  Traffic  Control  Center  portion  of 
a  large  Tactical  Advanced  Combat  Direction  and  Electronic  Warfare  System 
was  conducted  at  the  Fleet  Anti-Air  Warfare  Training  Center,  Point  Loma. 
Transfer  data  were  collected  on  the  DSS  Constellation  and  USS  Midway 
(Ref.  21).  The  data  indicate  that  increasing  the  time  spent  in  the 
trainer  results  in  increased  performance  at  sea.  The  findings  also  show 
that  team,  sub-team,  and  individual  capabilities  to  deal  with  recovery 
contingencies  and  emergencies  improve. 

Device  2F69B,  P-3A  Aircraft  Weapon  System  Trainer,  at  Patuxent  River, 
was  evaluated  using  training  squadron  ASW  crews  (Ref.  22).  Data 
collected  in  the  trainer  indicated  an  increase  in  performance  throughout 
five  Weapon  System  Trainer  (WST)  sessions.  This  increase  occurred 
despite  the  fact  that  instructor  aid  was  systematically  decreased  while 
at  the  same  time  the  level  of  task  difficulty  was  increased.  The 
improvement  in  task  performance  was  reflected  in  measures  of  accuracy 
and  efficiency.  The  accuracy  of  completing  such  mission  tauiks  as 
navigational  stabilization,  search,  localization,  and  attack  consistently 
improved  during  the  trainer  sessions.  Improved  accuracy  van  accompanied 
by  improved  efficiency.  As  the  students  progressed  through  V'T  training, 
they  reduced  the  time  spent  in  completing  each  evolution. 

To  demonstrate  the  extent  to  which  training  is  transfe.n-ed  to  the 
airborne  environment,  a  second  ph,»sa  of  the  study  was  directed  at 
obtaining  performance  measure*,  in  the  operational  setting.  Attempts 
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to  obtain  submarine  services  were  unsuccessful;  therefore,  the  transfer 
data  portion  of  the  study  was  conducted  using  surface  ships  as  targets. 

The  analysis  of  the  data  for  this  phase  of  the  study  has  not  been 
completed  as  of  this  writing. 

In  support  of  transfer  data  showing  the  value  of  training  devices, 
is  the  analysis  cf  training  situations  to  determine  tasks  that  can  and 
cannot  be  trained  in  the  training  device  and  in  the  operational  situation. 
An  example  of  such  ar.  analysis  will  be  givin  for  Device  14A2,  Surface 
ASW  Attack  Trainer,  for  which  a  study  of  the  retention  cf  skills  learned 
on  it  was  performed,  but  for  which  no  transfer  of  training  data  are 
available.  In  the  skill  retention  study  (Ref.  23),  performance  changes 
by  members  of  ASROC  teams  undergoing  training  at  Norfolk  were  measured, 
ar^  their  skills  reevaluated  at  periods  ranging  from  eight  to  sixteen 
weeks  after  training.  Two  rather  straightforward  conclusions  were 
reached.  One  is  that  the  trainees  do,  in  fact,  learn  in  the  trainer. 

The  other  is  that  they  rapidly  forget  what  they  have  learned  when  they 
go  to  sea.  It  was  concluded  that  shorebased  team  training  should  be 
made  a  regular  part  of  the  operating  schedule  of  ASROC-equirp ad  ships. 

The  consensus  expressed  was  that  Device  X14A2  practice  was  as  good  or 
better  than  at-sea  practice,  since  it  allows  for  multi-unit  problems 
and  unexpected  contingencies. 

TAEG  team  members  analyzed  the  tasks  that  can  be  performed  on 
Device  14A2  and  in  the  operational  situation.  They  concluded  that  if 
Device  14E19,  Basic  Operafcor/Team  Trainer  for  the  AN/SQS-26CX  Sonar, 
is  used  in  conjunction  with  Device  14A2,  everything  that  could  be 
trained  at  sea  could  be  conducted  in  the  trainer.  In  fact,  at  sea  it 
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is  extremely  difficult  to  conduct  other  than  basic  training  problems,  and 
intermediate  and  advanced  level  exercises  are  virtually  never  conducted. 
Intermediate  and  advanced  exercises  are  not  conducted  in  the  trainer 
either,  but  such  is  not  only  feasible,  but  highly  desirable.  This 
analysis  could  apply  in  general  to  various  attack  trainers.  Whether  this 
analysis  is  correct  will  be  tested  soon  with  the  planned  trailing  effective¬ 
ness  evaluation  of  Device  14A2  which  will  obtain  transfer  reasurements . 

A  recent  study  conducted  by  HumRRO  (Caro,  1970,  Ref.  24)  provides 
some  support  that  transfer  can  be  predicted.  An  approach  called  Equipment- 
Device  Task  Commonality  Analysis  was  developed.  It  identifies  the  stimuli 
and  responses  involved  in  the  operational  situation  and  the  training 
device,  and  then  determines  the  extent  to  which  the  stimuli  are  common 
to  both  the  operational  equipment  and  the.  device  It  then  looks  at 
response  commonality,  that  is,  the  extent  to  which  responses  made  in  the 
operational  environment  may  be  made  i::  the  training  device.  A  prediction 
of  positive  or  negative  transfer  traini vg  from  the  ‘device  to  the 
operational  equipment  is  made  from  the  i..rorraa'eion  concerning  stimulus 
and  response  commonality,  p.id  using  the  following  "principles"  of  transfer 
of  training  as  guidelines:  (1)  I-.  *;.ltive  transfer  will  occur  when  both 
stimuli  and  responses  ate  similar  in  the  training  situation  and  the 
operational  situation,  and  (2)  Negative  transfer  will  occur  when  the 
stimuli  are  similar  in  the  training  and  operational  situations,  but  the 
responses  to  the  similar  stimuli  are  different.  This  procedure  was  applied 
to  a  training  device  (Link  1-CA-i ,  a  fixed  wing  basic  instrument  trainer 
modified  to  a  helicopter  configuration)  whose  transfer  of  training  value 
had  been  previously  determined  empirically.  It  was  concluded  that 
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relatively  little  task  commonality  exists  between  it  and  the  operatiovml 
equipment  (TH-13T  helicopter).  Predominately  negative  transfer  was 
"predicted"  from  its  use,  a  "prediction"  that  was  supported  by  the  ear Her 
transfer  of  training  study.  (It  is  unfortunate  that  "prediction"  was 
not  done  prior  to  the  transfer  study.) 

The  relationship  between  training  effectiveness  and  cost  has  been 
discussed  by  many  authors  since  1954,  when  Miller  (Ref.  25)  introduced 
his  now  well-known  hypothetical  relationships  among  degree  of  fidelity 
of  simulation,  transfer  of  training,  and  simulator  cost.  His  curves 
depicted  an  increase  in  both  transfer  of  training  and  cost  with  in¬ 
creasing  degree  of  fidelity  of  simulation.  The  objective  is  to  find 
the  optimum  point  of  interaction  between  fidelity,  transfer  and  cost,  or 
in  other  words,  to  obtain  the  highest  degree  of  transfer  for  the  lowest 
possible  cost.  The  implication  is  that  it  is  necessary  to  make  compro¬ 
mises  between  economic  and  training  goals  when  selecting  training  media. 

The  problem  with  Miller's  relationship  is  not  so  much  the  hypo¬ 
thetical  shapes  of  the  curves,  but  the  implicit  assumption  that  training 
value  increases  as  a  function  of  fidelity  of  simulation.  Undoubtedly, 
increasing  fidelity  of  (engineering)  simulation  results  in  increased 
cost.  However,  Miller's  curve  which  shows  an  increasing  amount  of 
transfer  with  increasing  fidelity  of  simulation  is  disputable. 

A  recent  study  (Erickson,  et  al,  1972,  Re*.,  3  7)  stated,  "Only  a 
handful  of  studies  have  been  concerned  with  the  relationship  between 
fidelity  of  simulation  and  training  value.  Conflicting  results  have 
been  obtained!  in  some  studies  high  fidelity  simulation  produced 
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better  training,  while  in  others,  a  lesser  degree  of  fidelity  produced 
equally  good  training.  No  studies  have  been  reported  in  which  higher 
fidelity  is  associated  with  poorer  training."  The  authors  then  go  on 
to  recommend  high-fidelity  simulation. 

But  even  if  low  fidelity  simulation  resulted  only  in  training 
effectiveness  equal  to  that  obtained  from  high  fidelity  trainers,  the 
obvious  cost  effectiveness  requires  consideration  of  low  fidelity  of 
simulation  for  appropriate  tasks.  Many  of  the  studies  demonstrating 
that  low  fidelity  training  devices  are  as  effective  as  high  fidelity 
devices  or  operational  equipment  have  been  concerned  with  procedural 
tasks  In  which  every  motion  must  be  performed  in  sequential  order  (e.g., 
Grimsley,  1969,  Ref.  26;  Prophet  and  Boyd,  1970,  Ref.  27).  Several 
studies  were  done  using  flight  simulators  differing  in  their  degree  of 
fidelity  of  simulation  (Mahler  &  Bennett,  1949,  Ref.  13;  Wilcoxon,  et  al, 
1954,  Ref.  28;  Dough^ty,  et  al,  1957,  Ref.  29),  and  one  study  with 
five  uifferent  degrees  of  fidelity  in  submarine  contr61  (Newton,  1959, 
Ref.  30).  (The  latter  study,  however,  measured  transfer  not  in  an 
operational  situation  but  to  the  simulator  having  the  highest  degree, 
of  fidelity.)  fane rally,  despite  differing  degrees  of  fidelity,  there 
was  no  difference  in  transfer  effect  between  trainers.  It  is  contended 
by  the  present  report  that  training  effectiveness  is  more  a  function 
of  the  manner  in  which  the  trainer  is  used  than  of  the  fidelity  of  the 
trainer. 

The  goal  to  approach  complete  duplication  of  operational  equipment 
should  not  be  attempted  unless  a  training  situation  analysis  reveals 
its  necessity.  It  is  costly  to  do  so,  and  in  many  instances,  is 
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unnecessary  for  effective  training.  The  critical  element  of  training 
is  transfer  to  operational  performance  of  the  skills,  knowledges,  and 
attitudes  developed  in  the  training  situation.  This  depends  very 
heavily  on  how  the  training  device  is  used  rather  than  on  how  realisti¬ 
cally  the  device  is  designed. 

Maximizing  fidelity  is  a  very  costly  endeavor.  By  minimizing 
fidelity,  effective  training  can  be  provided  with  a  considerable 
reduction  in  cost,  thus  resulting  in  the  savings  of  resources  that  could 
then  be  used  elsewhere. 

CONCLUSIONS 

s  j 

In  1954,  Gagne  (Ref.  31)  stated  that  there  are  "a  number  of  studies 
on  the  ’effectiveness*  of  training  devices  which  are  generally  charac¬ 
terized  by  sound  but  unstartling  conclusions."  The.  findings  of  studies 
sines  that  rime  are  quite  similar,  so  again  his  statement  could  be 
made.  But,  the  view  held  here  represents  a  different  .interpretation  of 
the  findings,  namely,  that  trainees  can  learn  some  things  about  flying 
(or  other  operational  tasks)  while  they  are  practicing  in  training 
devices.  Though  the  research  results  themselves  are  not  necessarily 
"startling",  what  is  startling  is  the  resistance  to  substitution  of 
some  operational  training  time  by  training  devices.  It  was  not  until 
costs  of  non-revenue  training  flights  for  the  commercial  airlines  became 
so  tremendous  that  simulators  became  considered  anything  but  "supplements" 
to  flight  training.  As  a  result  of  this  demonstration  by  the  airlines 
of  the  feasibility  and  practicality  of  substitution  of  flight  time  by 
practice  in  training  devices,  the  military  should  boldly  adopt  the 
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policy  of  substitution  for  the  appropriate  portions  of  the  flight 
training  and  other  operational  training  syllabi. 

Training  effectiveness  evaluations  cf  training  systems  have  demon” 
strated  that  learning,  retention  and  transfer  occur  in  situations  where 
’'exact  simulation"  is  not  present.  (These  examples  do  not  necessarily 
violate  the  theory  of  "identical  elements,"  which  is  an  approach  of 
analyzing  transfer  in  terras  of  specific  elements  common  to  tasks.  All 
transfer  effects  cannot  be  related  to  an  analysis  of  specific  stimulus- 
response  relationships  (Ref.  32).)  We  may  generalize  from  the  examples 
of  training  effectiveness  evaluations  that  training  effectiveness  results 
not  from  attempting  to  approach  identity  of  task  elements,  but  from 
using  a  training  device  in  a  manner  that  permits  trainees  to  practice 
the  behaviors  critical  for  performance  in  the  operational  situation. 
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APPENDIX  A 

SUMMARY  OF  TRAINING  SYSTEMS 
EFFECTIVENESS  EVALUATIONS* 


*  Mr.  Joseph  A.  Puig  of  the  Human 
Factors  Laboratory  prepared  most 
of  this  Summary. 
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SOME  GENERAL  CONCLUSIONS 


-  Experiments  reveal  that  substantial  amounts  of  air  time  can  be 
substituted  for  by  simulator  time. 

-  Most  experimental  work  has  been  done  on  simple  aircraft  and  trainers. 

-  Different  kinds  of  flight  tasks  have  different  transfer  effects. 

-  The  level  of  simulation  and  kind  of  trainer  importantly  influence 
transfer. 

-  Careful  specification  of  both  trainer  and  operational  tasks  is 
necessary  if  transfer  is  to  occur. 

-  Motion  of  particular  kinds  affects  trainee  performance  and  transfer. 

-  Adding  motion  and  visual  displays  increases  fidelity  requirements. 
Coupling  of  these  is  a  major  issue. 

-  How  p,  device  is  used  may  irfluence  learning  and  transfer  to  a  greater 
degree  than  trainer  design. 

-  Differences  between  training  and  operational  equipment  are  necessary 
to  exploit  training  technology. 

-  A  precise  specification  of  tasks  and  measures  of  operational  transfer 
tasks  is  vital  to  effectiveness  evaluation. 
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